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With the passage of the No Child Left Behind Act 
(NCLB), state education agencies (SEAs) and local 
educational agencies (LEAs) have stepped up activities 
to broadly expand their data collection and reporting 
systems. The Data Quality Campaign (DQC) has advo- 
cated strongly for sfafes fo build robusf studenf-level 
longifudinal dafa sysfems based on sfandards and 
common definitions that address 10 essential elements. 

(See box, page 2.) 

fundamentals of a longifudinal dafa system to make it 
Although these elements are necessary, they are not become more than just a repository of vast amoimts of 

sufficient to make the states' data systems fully func- data. The data must be analyzed and widely used to 

tional or useful. The DQC also has identified seven make the large investments in such systems worthwhile. 

Data Are Only as Useful as People's Ability To Access, Understand and Use Them 


Highlights 

In this brief, find out more about: 

► Nine tips for making data accessible and useful. 

► Lessons learned from leading states for developing reporting 
and analysis tools. 


Before NCLB, mosf SEAs collected only aggregated or 
summarized data about school and student perform- 
ance from LEAs and then focused fheir energies on 
creating annual reports that were mandated by state 
and federal regulations. In facf, fhe reporting require- 
menfs from fhe U.S. Deparfment of Education (ED) 
generally came from specific program areas and offen 
were redundanf. 

Affer NCLB, an ED inifiative fo synfhesize reporfing 
requirements and streamline data submission processes 
— the Education Data Exchange Network — gained 
momentum and widespread support. SEAs, in turn, 
recognized the need to streamline their data collection 
and reporting processes as they expanded their data 


systems to include the student-level data needed to 
meet NCLB reporting requirements. 

As SEAs adapt to collecting a wider array of data at 
the student level, they also must adapt to the chang- 
ing demands for data and related analyses from more 
stakeholders. To date, SEAs have not always had the 
capacity, in staff and fechnology, to respond to both 
mandated reporting requirements and ad hoc requests 
from legislafures, researchers and reporfers, among 
others. 

Because it is not enough just to collect the data — the 
data must be used to make it worthwhile — more 
SEA staff and resources are needed in fhe new culfure 
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of data-driven decisionmaking. These staff and 
resources are vital to collect, clean, S5mthesize and 
share the data with other stakeholders (e.g., parents, 
educators, administrators and policymakers) so that 
all can be involved in the continuous improvement of 
student performance. Investments in data warehouses 
and reporting and analysis tools are the means for 
SEAs to be more responsive and get more reliable 
information to more people faster. 

Discussions about data warehouses and reporting and 
analysis tools so often occur at the same time that it is 
easy to think that they are one and the same. Although 


these two components of a robust data system go hand- 
in-hand, they require separate types of technology 
and staff expertise. 

A data warehouse is, in essence, a storage facility for 
many datasets culled from a variety of source files, 
such as student enrollment, program participation, 
graduation, state-level test data, teacher data and 
financial data. Reporting and analysis tools, however, 
are essentially the software programs written to calcu- 
late the statistics that stakeholders need to evaluate 
the performance of a student, school, district or state 
and produce reports (electronic or print) that answer 


10 Essential Elements of State Longitudinal Data 
Systems 

1 . A unique statewide student identifier that connects student data across 
key databases across years 

2. Student-level enrollment, demographic and program participation 
information 

3. The ability to match individual students' test records from year to year 
to measure academic growth 

4. Information on untested students and the reasons they were not 
tested 

5. A teacher identifier system with the ability to match teachers to 
students 

6. Student-level transcript information, including information on courses 
completed and grades earned 

7. Student-level college readiness test scores 

8. Student-level graduation and dropout data 

9. The ability to match student records between the P-1 2 and higher 
education systems 

10. A state data audit system assessing data quality, validity and reliability 


Fundamentals in Designing State Longitudinal Data 
Systems 

In addition to the 1 0 essential elements, states need to ensure that they take 

into account the following fundamental concepts in the construction of their 

longitudinal data systems: 

1. Privacy Protection — the assignment of unique student identifiers and 
guarantee that personally identifiable data do not become available to 
data users 

2. Data Architecture — the documentation and enforcement of rules 
regarding how data are coded, stored, managed and used 

3. Data Warehouse — a repository of data, such as student, staff, curricu- 
lum, facilities and finances 

4. Interoperability — the ability of different software systems from differ- 
ent vendors to share information without customized programming or 
data manipulation 

5. Portability — the ability to exchange student transcript information electron- 
ically across districts and states or between P-12 and postsecondary systems 

6. Professional Development around Data Processes and Use — the 

training of people charged with collecting, storing, analyzing and using data 

7. Researcher Access — the capacity and willingness to share data with 
researchers while meeting federal and state privacy regulations to enable 
research and evaluation studies 
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stakeholder questions. A separate implementation 
brief about the development of a data warehouse will 
be published alongside this brief. 

According fo fhe 2006 DQC survey,' 28 sfafes report 
having deployed a Web-based data access, analysis 
and reporting solution. State policymakers and educa- 
tors need a data system that makes it easy for users fo 
query those databases and review available reports to 
adapt policies, programs and teaching plans to the 
unique needs of fheir students. 

Reports and analytical tools are valuable only if they 
are used; therefore, they must be designed with the 
end-user at least in mind — but preferably wifh the 
end-user actually involved. 

Typical users of fhe reports and tools produced 
by the state include teachers, school and district 


administrators, state program area staff, and sfate- 
level policymakers, along with parents and students. 
Each group will have a different set of quesfions it 
wants answered, and standard reports produced by 
the state should address those questions. 

In addition, it is critical to display the data in such 
a way that the user can easily understand the report. 
This is often easier said than done. Consequently, 
the state must include representatives of each group 
in the design of fhe reports and allow for regular 
feedback to improve the reports and tools over time. 

It is common practice to work with information tech- 
nology (IT) staff in local districfs, but if the district 
has research staff — that is, people who are trained 
to ask research questions and analyze findings — it 
is important to include them in the design of the 
reports as well. 


Answering the Demand for Data Collection and Analysis 


In the late 1980s, before fhe education field knew how 
to use the full powers of IT, reports with analyzed 
data usually came in one of only two forms. Firsf, 
researchers in higher education and the upper eche- 
lons of P-12 administration studied questions and 
wrote reports that few people oufside of those ranks 
ever saw. Second, and far more well known, were the 
annual public reports of student achievement test 
scores by schools and districts. Test scores and rank- 
ings would appear in the local news and produce a 
brief flurry of discussion by the press and community. 
That would be the end of if. 

However, as information technologies expanded, edu- 
cators and researchers alike began exploring how to 
use the new technologies to answer questions that 
had dogged them for years. From statehouse fo 
schoolhouse, investments, initiatives and strategies 
had been implemented, but to what effect? Did the 


reading initiative, lowering of class size or new fesfs 
improve student achievement? If so, with whom? Are 
gains being sustained over time? And how should the 
answers be reported and put into the hands of fhe 
people who need to know? 

When NCLB became law, the modest IT offices of 
mosf SEAs suddenly were confronted with handling 
an even more daunting volume of new data and/or 
new data collection activities to comply with the law's 
mandates. 

Today, the technology exists to meet increasing data 
collection and reporting demands, but SEAs still need 
to expand the staff and resources fo be responsive fo 
these demands. This is equally true among LEAs. 


‘Data Quality Campaign/National Center for Educational 
Accountability, 2006 Survey of State Data Collection Issues Related 
to Longitudinal Analysis, 
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Next Generation of Data Use: Making Data Accessible and Useful 


Collecting and analyzing data are fairly straightforward 
activities when compared wifh designing reports and 
developing public information strategies for releasing 
whaf can be politically sensitive information. The public 
information office and IT office need fo work fogefher fo 
make fheir dafa useful fo fheir own agency, educators, 
researchers, policymakers and parenfs. Reporting and 
analysis sfrafegies will always be a moving fargef as fhe 
conversation abouf education dafa evolves and fhe pub- 
lic's questions and inferesfs shiff from focusing only on 
accounfabilify fo ensuring confinuous improvemenf. 
Every organization, education related or nof, benefifs 
from ongoing evaluation and analysis. 

Before responding fo an ad hoc requesf, SEAs musf 
review fhe requesf and desired producf fo ensure fhaf 
if meefs sfafe and federal privacy laws and fhaf fhey 
have bofh fhe fechnology capacify fo handle fhe 
requesf and fhe sfaff wifh fhe research and analyfical 
abilify fo produce fhe analyses needed. A dafa ware- 
house and sfandardized reporfing and analysis fools 
often make if easier fo be responsive because much of 
fhe dafa cleansing, merging and programming already 
has been performed. 

The DQC interviewed sfaff from Florida, Hawaii and 
Ohio fo ascerfain how each sfafe approached fhe 
design and developmenf of ifs reporfing and analysis 
fools and fo gafher lessons learned. These fhree sfafes 
were selecfed because of fhe varying ages of fheir dafa 
systems and fheir approaches fo reporfing and analy- 
sis fools. The common lessons learned include: 

1. Annual "report cards" and single-shot information 
briefs are usually summative tools. 

At a minimum, every sfafe has published NCLB 
reporfs online for fhe public view; fhese reporf cards 
range widely in qualify and user friendliness. Some 


have raw numbers loaded info fables fhaf are accom- 
panied wifh liffle explanafion and are nof parenf 
friendly. Af fhe ofher end of fhe specfrum are colorful, 
graphic, af-a-glance charfs fhaf use, for example, sfafe 
comparisons fo give fhe reader a confexf for under- 
sfanding fhe performance of an individual school. 

Few have longifudinal dafa fhaf show how fhe sfu- 
denfs have performed or changed over time. 

Alfhough schools are keenly aware of fheir public 
repufafions, fhe NCLB-mandafed reporfs show only 
fhe tip of fhe dafa iceberg. Below fhaf summafive fip 
lie defails fhaf can inform all levels of educafors as fo 
whaf is helping children learn versus whaf is ineffec- 
tive. The question confronfing all education sfake- 
holders is how fo organize fhe newly available dafa 
info regular cycles of reporfs fhaf supporf and inform 
fhe ongoing and confinuous improvemenf of every 
sfafe's schools and disfricfs. 

2. Ongoing inf ormation feedback loops serve 
decisionmaking. 

Arguably, fhe mosf imporfanf service fhaf educafional 
IT can provide is getting useful reporfs and informa- 
tion info fhe hands of classroom feachers, school com- 
munities and fhose who create policies fo supporf 
fhem. Sfafe dafa managers generally receive dafa 
requesfs from four primary groups of sfakeholders: 

► Schools and disfricfs; 

► Universifies and researchers; 

► Colleagues af fhe SEA and ofher governmenf agen- 
cies, such as Healfh, Social Services, Deparfmenf 
of Corrections, Deparfmenf of Labor and fhe 
Legislafure; and 

► Press. 
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Every stakeholder should have an opportunity to pro- 
vide feedback to the state to make sure that the 
reports are meeting their needs over time. Data man- 
agers continually are refining the access to the data, 
user friendliness of reports, protocols for control over 
the data's use, and the identification and use of new 
reporfing indicafors for evaluation activities and data- 
driven decisionmaking. 

In response to large volumes of armual data requests, 
each SEA must establish procedures for who in fhe 
agency will respond to requests and what type of 
product (report or dataset) will be provided. For 
example, in Hawaii, the Systems Accountability Office 
handles data requests — sometimes by forwarding 
predetermined reports and sometimes by sending 
datasets cleaned of idenfifying informafion. 

Florida's Office of Accountability, Research and 
Measurement has a customer relations manager who 
triages the requests. Normally, data requests of all 
kinds come in by e-mail. If the requestor wants some- 
thing that already exists, the manager directs the per- 
son to the appropriate source; otherwise, the manager 
forwards the state's Security and Access Policy to help 
the requestor understand what is and is not available 
and how to proceed. Similarly, Ohio's Office of 
Enterprise Applications has a data request manager 
whose unit operates in roughly the same way. They 
handle approximately 50,000 requests per year. 

The requests — as well as feedback from the schools 
and districts — help the data managers decide on any 
refinements to the system itself. For example, if a spe- 
cific type of request is made frequently, the SEA might 
develop a standardized report. Naturally, these refine- 
ments are decided in collaboration with the other SEA 
offices, such as assessment. 

3. Remember that reporting information is always 
political. 

Researchers tend to think of data as value-neutral or 
as objective pieces of information. However, when 


aggregated and put into report form, the topic of 
the report — such as student performance by sub- 
group — can become very sensitive, no matter how 
indisputable the facts. Not all answers or information 
are wanted or welcome. Even when the information 
will help with their work, people can easily become 
defensive, and reporfs wifhouf the necessary context 
and appropriate explanations or definitions can have 
serious political consequences. Data directors should 
work in partnership with the commimications staff to 
ensure the right messages are being crafted with the 
reporting and analysis tools. 

Florida provides an example of how powerful data 
analysis can be: Students in any Florida school that 
receives an "F" in its accountability report may 
transfer fo a charfer school if they choose. Recently, 
researchers wanted to test the widely held opinion 
that the charter schools were draining the more com- 
petent students from the regular public schools. If, in 
fact, the more competent students were the only ones 
leaving, the performance in the school from which 
fhey deparfed would decline fhe following year. Buf 
fhat was not the case. The competition brought on by 
the charter schools helped the public schools improve 
student performance. 

4. Guarantee confidentiality. 

Building a system based on historical student-level data 
automatically raises questions about the confidentiality 
of student records. Many of the state and federal 
laws governing privacy protections in education were 
enacted prior to the development of sfafe longitudinal 
data systems and, therefore, do not address all of the 
questions about how to best protect student privacy. 

The DQC's legal analysis provides a guide for stafe 
policymakers to ensure that the development and use 
of state longitudinal data systems are in compliance 
with the Family Educational Rights and Privacy Act.^ 

^Data Quality Campaign, Maximizing the Power of Education Data 
while Ensuring Compliance with Federal Student Privacy Laws: A Guide 
for Policymakers, 2007. 
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One of the key messages in this brief is that states 
need to ensure that routine procedures and processes 
are in place and followed to safeguard the personally 
identifiable information that is captured in a longitu- 
dinal data system. The bottom line, however, is that it 
is possible to promote wide-scale access and use of 
data while also protecting privacy. 

5. Minimize opportunities for misuse of data. 

As more data become available to the public through 
online sources, anyone with a computer can become 
an informal educational researcher, so the potential for 
the misuse of information becomes significant. 
Encouraging public access and use of data has to be 
balanced with the pitfalls of imcontrolled information. 

Most states are developing systems of role-based access 
to specific reports and queries. Each user is assigned a 
unique login name, password and role. Teachers, for 
example, have access to student-level data for only 
their students, while a school administrator has access 
to student-level data for only the students in his or 
her school. The same is true for district administrators. 
State policymakers, researchers and the press generally 
have access to aggregated data at the school and district 
levels and to specific types of reports and analyses, 
while parents generally have access to both the public 
aggregated data and their own child's records. 

6. Ensure data quality, validity and reliability. 

An SEA's credibility is established by avoiding incon- 
sistent reporting on similar indicators and by main- 
taining a well-defined, documented and cohesive data 
collection and reporting system for the entire agency. 
There should not be separate program-specific data 
collections and reports, imless they are conducted in 
coordination with the IT staff. 

To date, many states are still operating with siloed 
data systems across program-specific areas within the 
SEA. If, for example, teacher certification records are 
not collected with the principal teacher counts each 


year, an SEA might publish two different numbers as 
to how many teachers it has. Educators and the public 
become suspicious of the data in general when two 
different numbers surface for similar indicators. 

Partly to avoid the conflicting number problem, 
Florida was one of fhe firsf states to gather all of ifs 
information into a single data warehouse. But for 
years, the state was finding pockefs of unincorporated 
data tucked here and there within the SEA that were 
being gathered by different program areas according 
to their own rules, enabling different areas within the 
SEA to produce a number contradictory to one pro- 
duced by the data warehouse. 

7. Train the users of data to understand and use data. 

When the first NCLB "report cards" appeared, teachers 
and parents were almost entirely unschooled in data 
analysis. Since then, data literacy appears to be improv- 
ing. But every time a state changes a program, proce- 
dure or report, its successful implementation depends 
on good training of dafa users. Sfafe and local educa- 
tion agencies must teach their stakeholders how to read 
and interpret reports about their students and schools. 

Each year, staff in Hawaii's Student Assessment 
Section visit schools and districts to provide training 
on the state's standardized reports. They meet prima- 
rily with principals and administrators. Although 
some tend to just hand the data and reports off to 
their staff, many are becoming much more comfort- 
able with and adept at using the data. Increasingly, 
Hawaii's educators feel confident about backing up 
decisions with the data available from these reports. 

8. Adapt to changing culture of data-driven 
decisionmaking. 

As more data become infused throughout the education 
system, states must build the capacity of the SEAs, 
LEAs, parents and other stakeholders to use data on a 
daily basis for decisionmaking. Just as doctors base their 
diagnoses and prescriptions on research-based findings 
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and corporations analyze data to determine product 
and marketing effectiveness, so must educators, policy- 
makers and parents base their decisions on data. 

The way to do this is to make data readily accessible 
and useable via a wide array of diagnostics, reports 
and analyses geared toward answering specific 
questions about such things as academic growth, pro- 
gram cost effectiveness and teacher preparation. As 
changes in policies and programs are made because of 
early analyses, more questions will arise, and other 
data may be needed. SEAs must be responsive to 
ongoing changes in the culture of dafa, whether they 


are changes in technology, types of dafa fo collecf or 
requesfs for reporfs and analyses. 

9. Assess the role of vendors in developing sophisti- 
cated and useful reporting and analysis tools. 

Vendors are critical partners in the development of 
sfate longifudinal data systems, especially as many 
states struggle with budget shortfalls and limifed 
sfaffing. Alfhough many states try to build as much 
in-house capacity as possible, most states need to con- 
tract out some of fhe work. In some sfafes, vendors or 
confracfors are housed at the SEA and become part of 
its culture, sensitive to its unique needs and context. 


Developing Reporting and Analysis Tools: Three States' Stories 


Hawaii; Getting Longitudinal Data to Schools Quickly 

Unique among school systems in the United States, 
Hawaii is a statewide system, without districts or coun- 
ties. The islands each act as a region for organizational 
purposes buf have no independenf aufhority oufside of 
fhe sfafe's oversighf. The islands do nof keep studenf 
records apart from those of the SEA. With the records 
of their roughly 189,000 students already collected 
into a single dataset, Hawaii became an early convert 
to a system based on a rmique student identifier because 
fhe sysfem made mining fhe dafa easier. Thus, well 
before NCLB was passed, Hawaii had been analyzing 
longitudinal data because it could link student records 
from year fo year and from school fo school when 
studenfs moved or mafriculafed. 

Hawaii's SEA already had a long-standing relationship 
with a testing vendor. When faced wifh complying 
with NCLB, the vendor made it easy for fhe sfafe to 
add the necessary tests and avoid the massive start-up 
demands experienced by others. More complicated 
was creating an "Accountability Workbook," which 


outlined precisely how students and schools were to 
be held accoimtable for fheir performance. 

In 2003, Hawaii confracted wifh an individual con- 
fracfor fo create an Excel-based reporting tool 
designed to allow individual schools to analyze their 
own longitudinal student data. The following year, 
fhe SEA fook over fhe newly finished fool and has 
been refining bofh the tool and its in-house use. 

Prior to 2007, Hawaii created a CD every fall contain- 
ing the Excel-based, confidential student records for 
each individual school and distributed them by mail. 
The CD also contained tools necessary to allow dis- 
trict and school-level leaders to create their own 
reports. A large array of dropdown menus allowed 
users to generate their own reports. Sophisticated 
users also could export and manipulate the data in 
their own Excel spreadsheets. With considerable speci- 
ficity, the teacher or administrator could open histori- 
cal records for any child. Each file contained every 
year's data on the 37 target elements mandated by 
NCLB. School Improvement Teams or the region's 
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Central Office could ask and answer questions abouf 
how well fheir initiatives had performed over time. 

In fhe 2006-07 school year, an online porfal was creafed 
fhrough which fhe schools can download fheir own 
folder of dafa. 

Currenfly, Hawaii is committed mainly fo fhis one 
sfandardized yearly reporf because if has served fhe 
sfafe and fhe schools well as an information loop. The 
sfafe expecfs fo confinue fo refine if fo serve fhose 
educafors charged wifh improving fhe function of 
fheir schools. 

However, a differenf office wifhin fhe SEA, fhe Office of 
Information Technology Services, currenfly is building 
a more robusf dafa warehouse fhaf will link a wider 
variefy of indicators using fhe sfudenf identifier. This 
efforf should generate reporfs fhaf respond fo more 
general research questions, rafher fhan add fo fhe dafa 
the schools already receive. 

Florida; Tapping the Powers and Promise of a State 
Data Warehouse 

Florida is an especially good example of why defensi- 
ble dafa is critical. As wifh 23 ofher sfafes, Florida 
requires ifs sfudenfs fo pass high-sfakes fesfs, buf 
Florida requires sfudenfs fo pass fhese fesfs af two 
different points in their academic careers in order to 
progress to the next level. Third grade students must 
pass a reading test to matriculate to the 4th grade, and 
high school students must pass a 10th grade test to get 
a diploma. Thus, the state's data must be accurate 
because children's lives are directly affected. The 
schools, especially those whose results require them to 
be sanctioned by the state, need fast, meaningful and 
reliable dafa fo identify weaknesses and adjusf fheir 
programs accordingly 

Disfricfs send sfudenf dafa fo fhe sfafe, where fhey are 
assembled in a dafa warehouse. However, until 
recenfly even reporfs fo schools of longifudinal sfu- 


denf dafa were generafed af fhe sfafe level, from fhe 
dafa warehouse, wifhouf giving schools direcf access 
fo fhe year-fo-year, sfudenf-level source dafa. Each 
year, fhe SEA gave fhe disfricf ifs schools' assessmenf 
dafa on a CD. In fum, fhe disfricfs disfribufed each 
school's dafa fo fhe individual school. 

Buf fhis is changing, in 2007, Florida is conducfing 
a parallel release of ifs online fool and fhe CDs, and 
in 2008, if plans fo release resulfs only online. The 
sfafe has disfricf represenfafives who are in confacf 
wifh fhe Assessmenf Office and will help fhe schools 
and disfricfs wifh any problems fhey have wifh fhe 
online porfal, alfhough use of fhis porfal should be 
fairly infuifive. Bofh fhe CD and fhe online sife have 
reporfing fools of various kinds, allowing for users 
wifh varying levels of experfise. There are experf fools 
fhaf allow users fo create fheir own reporfs, buf fhere 
are also generic, predefined reporfs. For example, 
users can identify fhe charf fhey wanf and selecf fhe 
school plus fhe subpopulafion, and fhe dafa ware- 
house populafes fhe predefined graphics. 

Nof surprisingly, fhe qualify and size of Florida's 
dafasef makes ifs research capabilifies attractive fo ouf- 
siders. Universifies and federal organizations fhaf 
hack education issues often requesf permission fo con- 
ducf research of fheir own. These exfemal research 
requesfs go before a review process, and if fhe query 
involves cerfain specific populations — special educa- 
tion sfudenfs, for example — fhe review committee 
will involve fhe relevanf program office. Once fhe sfafe 
undersfands whaf fhe requesfor wanfs and ensures 
fhaf privacy concerns are mef, an agreemenf is made 
for fhe researchers fo share fheir resulfs wifh fhe sfafe. 
in ofher words, fhe attractiveness of Florida's informa- 
tion often means that researchers will partner with the 
state to spend their time crunching Florida's data for 
fhe benefif of bofh fhe researcher and fhe SEA. 

Hopefully, disfricfs and building-based educafors 
fhemselves also will become adepf af using fhe sfafe's 
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robust data. The Web-based portal greatly eases their 
access to the entire scope of Florida's data capabilities. 

The state also is developing a system to deliver test 
results directly to a teacher's desktop. The state is 
about halfway fhrough fhis five-year projecf. 

Ohio; Developing a Homegrown Data-Sharing Portal 
While Ensuring Privacy 

In 1997, fhe Ohio Office of Enferprise Applications 
(OEA) purchased Oracle software and continues to 
build its own software applications on that platform. 
Alfhough the Office of Accounfabilify designed fhe 
confenf for Ohio's firsf school accoimfabilify reporf 
card, fhe OEA builf fhe reporf ifself . In 1998, fhe reporf 
became sfandard for all schools and disfricfs. 

Each year, fhe reporfs were generafed fhrough fhe 
dafa warehouse by populating fhe fields of a reporf 
femplafe for each school and disfricf. The agency fhen 
prrnfed ouf enough copies for every parenf in fhe sfafe 
— close fo 2 million. Simulfaneously, fhe OEA was 
building an online version fo obviafe fhe need for 
paper and prinfing. The resulting online reporfs 
looked a little differenf fhan fhe prrnf version buf had 
fhe same information. 

Although the Ohio SEA was at the forefront of 
creafmg dafa warehouses, fhe warehouse's ufilify is 
limifed by fhe laws governing fhe unique sfudenf 
idenfifier — called fhe Sfafe Sfudenf Identifier (SSID). 
SSIDs can be used only wifhin fhe agency's primary 
dafa collection and reporting system, the Education 
Management and Information System, which 
includes the data warehouse. The SEA has strong 
research capabilities within a dataset, but it carmot 
track K-12 students into public higher education, 
unemployment, prison or the military. 

In fact, the SEA does not have any records with stu- 
dent names at all — only the identifiers. Due fo sfafe 


law, fhe SSID assignmenf sysfem is hosfed exfemally 
af a regional service cenfer, and fhe exfemal hosf is 
responsible for assisfing fhe disfricfs in assigning 
identifiers fo sfudenfs when fhey firsf enfer fhe sys- 
fem. Testing vendors send fesf resulfs direcfly fo fhe 
schools, which negofiafe direcfly wifh fhe vendor in 
fhe evenf of misfakes; fhe SEA is nof involved in 
school fesfing issues. Schools fhen send fhe cleaned 
dafa fo fheir disfricf offices, which each have a 
"regional enfify" called fhe Information Technology 
Cenfer (ITC), fo manage fheir dafa. The ITC confirms 
each child's code and fhen strips out the child's name 
and identifying information before sending fhe dafa 
on fo fhe SEA. 

Resfricfions on fhe use of identifiers make for some 
awkwardly execufed research, alfhough fhe SEA man- 
ages fo fulfill mosf dafa requesfs. According fo fhe 
agency, ifs work could be more efficienf and eleganf, 
buf fhe resfricfions have forced some creative solu- 
fions, such as fhe powerful Dafa-Driven Decisions for 
Academic Achievemenf sysfem, or D3A2. (See 
www.d3a2.org/ fools.asp.^) 

Concerned fhaf if was spending foo much fime man- 
aging dafa af a high level — cooperating wifh federal 
requiremenfs, sfafe compliance issues and bird's-eye 
research — OEA inifiafed fhe work fhaf became D3A2 
fo give more help directly to the teachers and the stu- 
dents. (Now, the agency does not host or oversee it.) 

Through D3A2, starting in fall 2006, feachers in 12 
pilof disfricfs were issued passwords fhaf allow fhem 
fo see dafa relating only fo fheir own sfudenfs. 
Adminisfrafors can see fhe dafa for fheir school only. 
The passwords also fell fhe sysfem adminisfrafors 
who is looking af fhe dafa, an added measure of secu- 
rify for sfudenf privacy. 


“Data Quality Campaign, Data Use Drives School and District 
Improvement, Quarterly Issue Meeting: September 25, 2006, 
Washington DC, www.DataQualityCampaign.org/activities. 
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Now, educators can use their own computers to 
search a vast database that gives them not only results 
for each of their students, but also how the child 
scored on each individual test item. With its intention- 
ally easy-to-use interface, teachers can manipulate 
data to see how their African-American students per- 
formed on a single test question, for example. Or 
they can determine which students have not grasped 
a particular skill. Such data will help teachers and 
school commimities build on strengths and pinpoint 
weaknesses in their program. These data give them 
hard evidence they can use to back up a decision, 
adjust a program or advocate for a certain change. 


Furthermore, D3A2 also offers a "Content Repository" 
— a vast array of "lesson plans, activities, assessment 
items and instructional resources in a variety of for- 
mats from some of Ohio's most trusted content 
providers. These resources are aligned to Ohio state 
standards and searchable either by the standards or 
keywords," according to the program's Web site. A 
large committee of stakeholders, led by the SEA, 
regularly vets new content offerings before they are 
included. As of 2007, everything offered is free — 
shareware and materials that can be copied and dis- 
tributed. Such a tool has great potential to improve 
classroom practice and, in time, student achievement. 


Conclusion 


A consistent refrain is heard in statehouses, district 
offices and school staff meetings: "We need quality 
education data to do our jobs." Good information is 
critical to both the processes and the outcomes of a 
high-performing education system, and increasing 
numbers of voices, both inside and outside the sys- 
tem, are calling for this issue to be a national priority. 
It is widely acknowledged that vital policy conversa- 
tions now under way — conversations about increas- 
ing the rigor and relevance of high school, improving 


teacher quality, promoting higher graduation rates 
and reducing achievement gaps — cannot be success- 
ful unless they are informed by reliable longitudinal 
data. 

We must build the will and understanding to use this 
information for policy and program decisions. Designing 
and deploying sophisticated and user-friendly report- 
ing and analysis tools is vital to informing these 
policy conversations. 
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Resources 


American Association of School Administrators. 
www.aasa.org/links/content.cfm?ItemNumber=2626 

Consortium on School Networking. 
www.3d2know.org 

Mid-continent Research for Education and Learning. 
www.mcrel.org / PDF / LeadershipOrganization 
Development / 503 1 TG_dataf olio .pdf 

National School Boards Association. 
www.Nsba.org/ site/page.asp?TRACKID=&DID= 
8648&CID=402 

Wayman, Jeffrey C., 2005. Involving Teachers in Data- 
Driven Decision Making: Using Computer Data Systems 
to Support Teacher Inquiry and Reflection, Journal of 
Education for Students Placed at Risk, Vol. 10, 

No. 3, pp. 295-308. 

www.leaonline.com/doi / abs/ 10.1207 / sl5327671 
esprl003_5?cookieSet=l&joumalCode=espr 


State Contacts: 

Florida Department of Education 

Jeff Sellers, Assistant Deputy Commissioner of Accountability, Research, 
and Measurement, jeff.sellers@fidoe.org 

Hawaii Department of Education 

Robert McClelland, Director of Systems Accountability, 
robert_mcclelland@notes.k12.hi.us 

Ohio Department of Education 

Michael Carmack, Director of Office of Enterprise Application, 
michael.carmack@ode.state.oh.us 


Vendors used by states for reporting and analysis tools 
as reported on 2006 Data Quality Campaign survey: 

Crystal Reports, Cognos, Edsmart, eSchoiar, Grow Network, IBM, Infinite 
Campus, Microsoft SQL, Microstrategy, PCG, Pearson, Powerplay Tools, 
SAS, Siigo Computer Services and Tetra Data." 

'The Data Quality Campaign does not endorse any vendor; this Is a list of vendors 
used by states as displayed on the DQC Web site (www.DataQualltyCampalgn.org/ 
survey_results/vendors_ 2006 .cfm). 
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Using Data To [mprove Student Achievement 


www.DataQualityCampaign.org 


The Data Quality Campaign is a national, collaborative effort to 
encourage and support state policymakers to improve the collection, 
availability and use of high-quality education data and to implement 
state longitudinal data systems to improve student achievement. The 
campaign aims to provide tools and resources that will assist state 
development of quality longitudinal data systems, while providing a 
national forum for reducing duplication of effort and promoting 
greater coordination and consensus among the organizations focus- 
ing on improving data quality, access and use. 

Managing partners of the Data Quality Campaign include: 

► Achieve, Inc. 

► Alliance for Excellent Education 

► Council of Chief State School Officers 

► Education Commission of the States 

► The Education Trust 

► National Association of State Boards of Education 

► National Association of System Heads 

► National Center for Educational AccoimtabiUty 

► National Center for Higher Education Management Systems 

► National Governors Association Center for Best Practices 

► Schools Interoperability Framework Association 

► Standard & Poor's School Evaluation Services 

► State Educational Technology Directors Association 

► State Higher Education Executive Officers 


Endorsing partners of the Data Quality Campaign include: 

► ACT 

► Alliance for Quality Teaching 

► American Association of Colleges for Teacher Education 

► American Association of State Colleges and Universities 

► The American Board for Certification of Teacher Excellence 

► APQC 

► The Business Higher Education Forum 

► Center for Teaching Quality 

► College Summit, Inc. 

► Consortium for School Networking 

► Educational Policy Institute 

► ETS 

► GreatSchools 

► Institute for a Competitive Workforce 

► Institute for Educational Leadership 

► Jobs for the Future 

► League of Education Voters Foundation 

► Learning Point Associates 

► National Alliance for Public Charter Schools 

► National Association of Secondary School Principals 

► National Council for Accreditation of Teacher Certification 

► National Education Knowledge Industry Association 

► Pathways to College Network 

► Postsecondary Electronic Standards Council 

► Roads to Success 

► The Southern Regional Education Board 


For more information about the Data Quality Campaign, please visit www.DataQualityCampaign.org. 
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